Skip to content

[codex-analytics] add steering metadata#16706

Open
rhan-oai wants to merge 3 commits intomainfrom
pr16706
Open

[codex-analytics] add steering metadata#16706
rhan-oai wants to merge 3 commits intomainfrom
pr16706

Conversation

@rhan-oai
Copy link
Copy Markdown
Collaborator

@rhan-oai rhan-oai commented Apr 3, 2026

@rhan-oai rhan-oai changed the base branch from pr16659 to main April 7, 2026 21:53
@rhan-oai rhan-oai changed the base branch from main to pr16659 April 7, 2026 22:01
@rhan-oai rhan-oai changed the base branch from pr16659 to main April 7, 2026 22:08
@rhan-oai rhan-oai changed the base branch from main to pr16659 April 7, 2026 22:10
@rhan-oai rhan-oai changed the base branch from pr16659 to main April 7, 2026 22:14
@rhan-oai rhan-oai changed the base branch from main to pr16659 April 7, 2026 22:23
@rhan-oai rhan-oai changed the base branch from pr16659 to main April 7, 2026 22:53
@rhan-oai rhan-oai changed the base branch from main to pr16659 April 7, 2026 23:00
@rhan-oai rhan-oai changed the base branch from pr16659 to main April 7, 2026 23:17
@rhan-oai rhan-oai changed the base branch from main to pr16640 April 7, 2026 23:37
out: &mut Vec<TrackEventRequest>,
) {
let Some(RequestState::TurnSteer(pending_request)) =
self.requests.remove(&(connection_id, request_id))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to do this for TurnStart too? in case a JSON RPC error gets thrown for turn/start

if error.message.starts_with("expected active turn id ") {
return Some(TurnSteerRejectionReason::ExpectedTurnMismatch);
}
if error.message == "input must not be empty" {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, should we figure out a less brittle way to do this? maybe define shared constants for the strings?

.get("codexErrorInfo")?
.get("activeTurnNotSteerable")?
.get("turnKind")?
.as_str()?;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm yeah... ideally there's a way to match on CodexErrorInfo instead of JSONRPCErrorError because then you'll have stronger type guarantees

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or maybe you can just cast data.get("codexErrorInfo") back to CodexErrorInfo and then pull out turnKind

ErrorResponse {
connection_id: u64,
request_id: RequestId,
error: JSONRPCErrorError,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think of adding something well-typed here actually, so we don't have to do string parsing on JSONRPCErrorError?

maybe:

ErrorResponse {
    connection_id: u64,
    request_id: RequestId,
    error: JSONRPCErrorError,
    type: Option<JsonRpcErrorType>,
}

where type is something like:

pub enum JsonRpcErrorType {
    Turn(TurnRequestError),
    Input(InputError),
}

pub enum TurnRequestError {
    NoActiveTurn,
    ExpectedTurnMismatch,
    ActiveTurnNotSteerable { turn_kind: NonSteerableTurnKind },
}

pub enum InputError {
    Empty,
    TooLarge { max_chars: usize, actual_chars: usize },
}

something like that? just brainstorming

Copy link
Copy Markdown
Collaborator

@owenlin0 owenlin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like there's a clippy error:

--> analytics/src/analytics_client_tests.rs:306:1
|
153 | fn sample_runtime_metadata() -> CodexRuntimeMetadata {
| ---------------------------------------------------- previous definition of the value sample_runtime_metadata here
...
306 | fn sample_runtime_metadata() -> CodexRuntimeMetadata {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ sample_runtime_metadata redefined here

.count()
}

fn now_unix_seconds() -> u64 {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: i've seen this function defined in a bunch of places at this point, should we centralize it somewhere? can do in a followup

message: "expectedTurnId must not be empty".to_string(),
data: None,
};
self.track_error_response(&request_id, &error, /*error_type*/ None);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm let's revert this? i don't think it's worth tracking this one

.as_secs()
}

fn rejection_reason_from_error_type(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a lot of boilerplate, any way we can eliminate this?

Copy link
Copy Markdown
Collaborator

@owenlin0 owenlin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments but preapproving

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants